Semiparametric Methods in the Estimation of Tail Probabilities and Extreme Quantiles

Title of dissertation: SEMIPARAMETRIC METHODS IN THE ESTIMATION OF TAIL PROBABILITIES AND EXTREME QUANTILES Lemeng Pan, Doctor of Philosophy, 2016 Dissertation directed by: Professor Benjamin Kedem Department of Mathematics In quantitative risk analysis, the problem of estimating small threshold exceedance probabilities and extreme quantiles arise ubiquitously in bio-surveillance, economics, natural disaster insurance actuary, quality control schemes, etc. A useful way to make an assessment of extreme events is to estimate the probabilities of exceeding large threshold values and extreme quantiles judged by interested authorities. Such information regarding extremes serves as essential guidance to interested authorities in decision making processes. However, in such a context, data are usually skewed in nature, and the rarity of exceedance of large threshold implies large fluctuations in the distribution’s upper tail, precisely where the accuracy is desired mostly. Extreme Value Theory (EVT) is a branch of statistics that characterizes the behavior of upper or lower tails of probability distributions. However, existing methods in EVT for the estimation of small threshold exceedance probabilities and extreme quantiles often lead to poor predictive performance in cases where the underlying sample is not large enough or does not contain values in the distribution’s tail. In this dissertation, we shall be concerned with an out of sample semiparametric (SP) method for the estimation of small threshold probabilities and extreme quantiles. The proposed SP method for interval estimation calls for the fusion or integration of a given data sample with external computer generated independent samples. Since more data are used, real as well as artificial, under certain conditions the method produces relatively short yet reliable confidence intervals for small exceedance probabilities and extreme quantiles. This dissertation is organized as follows: In Chapter One, an overview of Extreme Value Theory will be given, and the existing methods for exceedance probability and extreme quantile estimation in EVT will be presented in some detail. Chapter Two introduces some necessary background about the Density Ratio Model. In Chapter Three, the idea of out of sample fusion (OSF) and repeated out of sample fusion (ROSF) are reviewed. We will show how to estimate tail probabilities and construct confidence intervals through OSF and ROSF. Results from extensive simulation studies are presented to demonstrate the performance of the proposed model when the underlying sample is from a highly skewed distribution. The results are compared with those obtained by EVT, and other well known methods. In Chapter Four, how extreme quantiles are estimated based on ROSF is presented with results from simulation studies. In Chapter Five, applications of the proposed method to real data problems in food safety and a clinical trial will be given. Finally, asymptotic theorems and results for quantiles under the density ratio model appear in the appendix. SEMIPARAMETRIC METHODS IN THE ESTIMATION OF TAIL PROBABILITIES AND EXTREME QUANTILES